Learning simulator for quantum-algorithm design 
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We propose a general-purpose method of machine learning for quantum- algorithm design. The 
method is of using a quantum- classical hybrid simulator, where a "quantum student" is being taught 
by a "classical teacher." In other words, in our method, the learning system is supposed to evolve 
into a quantum algorithm for a given problem, assisted by classical feedback. Our method is 
applicable to design, in principle, every quantum oracle-based algorithm. As a case study we show 
that the quantum algorithms solving the Deutsch-Jozsa problem can be faithfully learned. Even 
more remarkable is that the learning time is proportional to the square root of the total number of 
parameters instead of the exponential dependance found in the classical cases. 

PACS numbers: 03.67.Ac, 07.05.Mh 



Introduction. - Quantum information has seen explo- 
sive growth in recent years, as a more powerful general- 
ization of classical information theory [1]. In particular, 
quantum computation has received its momentum from 
the quantum algorithms that outperform their classical 
counterparts [2|-[5|. Thus, the development of quantum 
algorithms is one of the most important areas of com- 
puter science. However, unfortunately, recent research on 
quantum- algorithm design is rather stagnant, compared 
to other areas in quantum information, as new quantum 
algorithms have scarcely been discovered in the last few 
years [6|. We believe that this is due to the fact that we 
- the designers are used to classical logic. Thus we think 
that the quantum- algorithm design should turn towards 
new methodology, different from the current approach. 



Machine learning is a well-developed branch of artifi- 
cial intelligence and automatic control. Although "learn- 
ing" is often thought of as a uniquely human trait, a 
machine being given feedback (taught) can improve its 
performance (learn) in a given task 0, Q. In the last 
decades, there has been a growing interest not only in the 
theoretical studies but also in a variety of applications 
of the machine learning. Recently, many quantum im- 
plementations of machine learning have been introduced 
to achieve better performance for quantum tasks [qI-Ii^. 
These works motivate us to look at machine learning as 
an alternative approach for quantum-algorithm design. 

Keeping our primary goal in mind, we ask whether a 
quantum algorithm can be designed by the machine that 
also implements it. Based on this idea, we consider a 
machine which is able to learn quantum algorithms in 
a real experiment. Such a machine may discover solu- 
tions which are difficult for humans to find because of 
our classical way of thinking. Since we can always simu- 
late a quantum machine on a classical computer (though 
not always efficiently) we can use such simulations to de- 



sign quantum algorithms without the need for a program- 
able quantum computer. This classical machine can thus 
be regarded as a simulator that learns a quantum algo- 
rithm, so-called learning simulator. The novelty of such a 
learning simulator is in its capabilities of "learning" and 
"teaching". With regard to these abilities, we consider 
two internal systems: One is a learning-system ("stu- 
dent" say), and the other is a feedback-system ("teacher" 
say). While the standard approach is to assume that both 
of student and teacher are quantum machines here we use 
a quantum- classical hybrid simulator such that the stu- 
dent is a quantum and the teacher a classical machine. 
Such a hybridization is easier and more economical to 
realize if any algorithms are able to be learned. 

In this Letter, we employ the aforementioned simula- 
tor for quantum- algorithm design. Our simulator is of 
general purpose and can be used to learn any quantum 
algorithm with classical feedback. Nevertheless, to study 
its behavior we need to choose a particular one. For a 
problem to be solved we choose an oracle decision prob- 
lem, called Deutsch-Jozsa problem. For the feedback 
system, we adopt a differential evolution method fl5|. 
To show that our simulator can faithfully learn quantum 
algorithms we use Monte-Carlo simulations. The algo- 
rithms found are equivalent, but not exactly equal, to 
the original Deutsch-Jozsa algorithm. The most remark- 
able result is that the learning time is proportional to 
the square root of the total number of parameters to ex- 
plore, instead of the exponential tendency found in the 
classical cases. This result will work in synergy with the 
findings that the size of the parameter space can be sig- 
nificantly smaller for quantum algorithms than for their 
classical counterparts [16]. Therefore, the quantum ma- 
chines have a double advantage: Not only their learning 
time scales more favorably with the size of the space but 
this size is smaller to begin with. To our knowledge, this 
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FIG. 1: (Color Online) Basic architecture of the simulator. 
This simulator is a quantum-classical hybrid which consists 
of "P-?7-M" quantum building block assisted by classical F. 
The classical channels Cmf and Cfu enable one-way commu- 
nication from M to F and from F to U. 
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FIG. 2: (Color Online) Architecture of a simulator for 
Deutsch-Jozsa problem, where the unitary operation U con- 
sists of three sub-operations (See the text). Ui and U3 are 
the parts of the algorithm optimized by the learning proce- 
dure. U2 is the oracle which encodes the constant or balanced 
function. 



is the first observation of the effectiveness and the con- 
vergence in the machine learning for quantum- algorithm 
design. We note that the presented method is also aimed 
at a real experiment, in contrast to the techniques of 
Refs. [13, [ill. 

Basic architecture of the learning simulator. - Before 
discussing the details of learning simulator, it is impor- 
tant to have an understanding of what machine learning 
is. A typical task of machine learning is to find a function 
f{x) = tx for the input x and the target tx based on the 
observations in supervised learning or to find some hid- 
den structure in unsupervised learning [7, 8]. The main 
difference between supervised and unsupervised learning 
is that in latter case the target tx is unknown. Through- 
out this Letter, we consider a supervised learning where 
the target tx is known. 

We now describe the basic elements of the learning 
simulator. The simulator consists of two internal parts. 
One is the learning-system which is supposed to even- 
tually perform a quantum algorithm, and the other is 
the feedback-system responsible for teaching the former. 
The learning-system consists of the standard quantum 
information-processing units: The preparation device P 
to prepare a pure quantum state, the operation device U 
to perform an unitary operation, and the measurement 
device M. On the other hand, the feedback-system is 
classical as it is easier and less expensive to realize in 
practice. Furthermore, by employing the classical feed- 
back, we can use a well-known (classical) learning algo- 
rithm whose performance has already been proved to be 
reliable. Recently, a scheme for machine learning involv- 
ing a quantum feedback has been reported [l^, but the 
usefulness of the quantumness has not been clearly eluci- 
dated, even though their results are meaningful in some 
applications. Moreover, it is unclear yet whether any 
classical feedback is applicable to the quantum-algorithm 
design. Consequently, it is preferred to use the classi- 



cal feedback in this work. In the sense, this simulator 
is a quantum- classical hybrid. The feedback-system is 
equipped with a main feedback device F which involves 
the classical memory S and the learning algorithm A. 
S records the control parameters of U and measurement 
results of M. A corresponds to a series of rules for updat- 
ing U. In Fig. [TJ the schematic diagram of the learning 
simulator is presented. 

We briefly illustrate how such a simulator performs the 
learning. To begin, provide a set of K input-target pairs, 
T = {(xi, /(xi)), (x2, /(X2)), . . . , {xK, /(xk))}, where / 
is a function that transforms the inputs Xi into their tar- 
gets. Our goal is to find /. To this end, a supervisor 
sends T to the simulator. Then, it starts learning. First 
a state |^in) is prepared in P and transformed to |^out) 
by U. Then M performs measurement on |^out)- The 
measurement result is delivered to F through Cmf- Fi- 
nally, F updates U based on A. Basically, the learning is 
just the repetition of these three steps. When the learn- 
ing is completed, we obtain P-U-M devices to implement 
/ by simply removing F. The supervisor can analyze 
if the found P-U-M provides any speedup or saves any 
computational resources. 

Here, we clarify that the input information in T and 
the measurement results are classical. Nevertheless, the 
simulator is supposed to exploit quantum effects in learn- 
ing, because the operations before measurement are all 
quantum. This assumption is supported by recent theo- 
retical studies that show the improvement of the learning 
efficiency by using quantum superposition lH, [2o| . 

Construction of the learning simulator. - The general 
design of the learning simulator depicted in Fig. [T] works 
fine for problems where the input is classical (e.g. num- 
ber factorization). However, in the problems involving 
oracle, the input is the oracle itself and, by definition, it 
is a (unitary) transformation rather than a string of bits. 
To allow for the input in the form of an unitary matrix 
we need to refine our simulator a little (but let us stress 



3 



that this does not mean that our method is not general). 
The refined version depicted in Fig. [2] allows the simu- 
lator to learn any algorithm with oracle reference. The 
difference in the learning simulators stems directly from 
the formulation of the problems. 

The most important aspect in the refined learning sim- 
ulator is the decomposition of U. In order to deal with 
both classical and quantum information, we divide U into 
three sub-devices, such that 



t/tot = %(p3)t/2(p2(xO)/7i(pi), 



(1) 



where /Ttot is total unitary operator, and Uj {j = 1,2,3) 
denotes the unitary operator of jth sub-device. U2 is the 
part that encodes the (classical) input Xi as the oracle. 
Ui and Us are n-qubit controllable unitary operators. By 
controllable we here, and throughout the paper, mean 
that they can be changed by the feedback. 

The unitary operators can generally be parametrized 

as 



2l|,l22| 



U{p) = exp {-ip^G) 



(2) 



where p = (pi,p2, • • • ^Pd'^-i)^ is a real vector in {(P — 
l)-dimensional Bloch space for d = 2"^, and G = 
(^1, ^2, • . • ^d^-i)"^ is a vector whose components are 
SU((i) group generators. In a real experiment, such form 
of unitary operation is realized with control parameters 
Pj G [— 7r,7r] [23|, [2^. In that sense, we call p a control 
parameter vector. Here P2(^i) is determined by the in- 
puts in T. In such setting, we expect that our simulator 
learns an optimal set of {pi, pa}, so that Ui and Us come 
to solve a given problem. 

We emphasize that the design depicted in Fig. [2] is for 
general purpose. The simulator is actually well-suited 
to learn even iterative algorithms, such as Grover's [HI. 
We envision using our simulator as follows: In the first 
stage, apply Ui to an input-state, then U2 which is a non- 
trivial operation, say oracle, and finally Us to generate 
an output state. The feedback-system updates Ui and 
Us. Then, after a certain number of iterations which do 
not lead to any improvements, our simulator goes to the 
second stage, where the output state is fed back to be 
the input state to apply U1-U2-US again. Therefore, in 
the second stage, the oracle is referenced twice. If it fails 
again it will try to loop three times at the third stage. 
By some number of stages, there will be enough oracle 
references to solve the problem. In such way, our sim- 
ulator can learn every known quantum algorithm (ssj . 
without adopting any additional sub-devices and alter- 
ing the structure in a real experiment Thus, the 
scalability for the size of the search space is only con- 
cerned with the number of control parameters in Ui and 
Us, given by D = 2 x - 1), where d = 2^. 

Application to Deutsch-Jozsa problem. - As a case 
study, consider an n-bit oracle decision problem, called 
Deutsch-Jozsa (DJ) problem. The problem is to decide 



if some binary function x:{0, 1}^ {^A} is constant {x 
generates the same value or 1 on every input) or bal- 
anced {x generates on exactly half of the inputs, and 
1 on the rest of the inputs) [2|, [3j . On a classical Turing 
machine 2^~^ + 1 queries are required to solve this prob- 
lem. If we use a probabilistic random classical algorithm, 
we can determine the function x with a small error, less 
than 2-^ by g queries 

On the other hand, DJ quantum algorithm solves the 
problem by only single query [271, l28|. The DJ quan- 
tum algorithm runs as follows: First, apply H^^ on the 
input state |^in) = |00 • • • 0), then Ux to evaluate the in- 
put function, and finally H^'^ again to produce an output 
state l^out)- Here, H is Hadamard gate which transforms 
the qubit states |0) and |1) into equal superposition states 
H |0) = (|0) + |1)) /^/2 and ^ |1) = (|0) - |1)) /V^ re- 
spectively. Ux is the function-evaluation gate that cal- 
culates a given function x. It is defined by its ac- 
tion, Ux \k1k2 ■ ■ - kn) = e*''^(^i^2---fer^) \k1k2 ■ ■ • kn), where 
kik2 ■ ■ ■ kn G {0, 1}^ is the binary sequence of the com- 
putational basis. Then, the output state is given as 



|^out(:^^i)) = 



±|oo---o), if XieC 

± \z1Z2 Zn) , if Xie B 



(3) 



where C and B are the sets of constant functions and 
balanced functions, respectively, and the binary compo- 
nents Zj G {0, 1} (j = 1,2,..., n) depend on the (^^2) 
balanced functions (excepting that Zj = for all j). In 
the last step, von-Neumann measurement is performed 
on the output state. The corresponding measurement 
operator is given by M = |00 • • • ) ( 00 • • • 0|. The other 
projectors constituting the observable are irrelevant be- 
cause we are interested only in the probabilities associ- 
ated with the first case 

Pc = (^out(^z)l M l^out(^z)) = 1, if X, e C, (4) 

and the second case 

Pb = (^out(^^)l M \^out{xi)) = 0, if X, e B. (5) 

Therefore, we can determine by only single oracle query 
whether the function x is constant or balanced. 

We are now ready to apply our method to the DJ prob- 
lem. To begin, supervisor prepares the set of input-target 
pairs, T = {{xi, f{xi))\f{xi) = 'c' if Xi e C and f{xi) = 
'b' if Xi G B}. The learning simulator is to find the 
"functional" / now as adjusting Ui and Us- The in- 
put functions Xi are encoded in P2{xi) of U2. Then P 
prepares an arbitrary initial state |^in) and M performs 
the measurement on each qubit. Here we introduce a 
function to apply a measurement result to one of the 
targets (in our case, 'c' or 'b'). We call this "interpre- 
tation function." For a sake of convenience, we consider 
a Boolean function that transforms the measurement re- 
sult zo^i • • • to (equivalent ly, 'c') only if Zj = for all 
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j = 0, 1, . . . , n, and otherwise 1 (equivalently, 'b'). One 
may generahze the interpretation function to a function 
{0,1}^ {0,1}^, if interested in any other problems 
that contain many targets less than 2^. It can be real- 
ized as done for a classical reversible circuit with n + m 
channels and Toffoli-gates [29|. Note that the interpre- 
tation function is also to be learned, because, in general, 
any a priori knowledge of the quantum algorithm to be 
found is completely unknown. 

Learning algorithm. - One of the most important parts 
in our method is choosing a learning algorithm A. Effi- 
ciency and accuracy of machine learning are heavily in- 
fluenced in general by the algorithm chosen. We employ 
so-called "differential evolution", as it is known as one 
of the most efficient optimization methods [l5|. We im- 
plement the differential evolution as follows. To begin, 
we prepare Np^p sets of the control parameter vectors: 
{Pi,i, P3,i} = 1, 2, • • • , iVpop). Thus we have 27Vpop pa- 
rameter vectors in total. They are chosen initially at ran- 
dom and recorded on S in F. [L.l] Then, 2A^pop mutant 
vectors iyk,i are generated for Uk {k = 1,3), according 

to = Pk,a + W{pk,b - P/e,c), whcrC Pk,a, P/e,6, and 

Pk,c are randomly chosen for a,6, c G {1,2,--- ,A/'pop}. 
These three vectors are chosen to be different from each 
other, for that A/'pop > 3 is necessary. The free parame- 
ter called a differential weight, is a real and constant 
number. [L.2] After that, all 2A^pop parameter vectors 
Pk,i = (Pfc,i,P/c,2, • • • ,Pk,d^-i)I are reformed to trial vec- 
tors T/e,i = (r/e,i,r/e,2, • • ' ,rfe,d2-i)f by the rule: For each 
J, 



TkJ ^ PkJ if Rj > Cr, 

otherwise. 



Tkj ^ ^k 



(6) 



where Rj G [0, 1] is a randomly generated number and 
the crossover rate Cr is another free parameter in be- 
tween and 1. [L.3] Finally, {ri^i^r^^i} are taken for 
the next iteration if Uiiri^i) and U^ir^^i) yield a larger 
fitness value than that from /7i(pi,i) and ^(ps^i); if not, 
{Pi,i7P3,i} are retained. Here the fitness is defined by 



Pc,i + (1 - PB,i) 



(7) 



where Pc^i and Pb^i are measurement probabilities for 
z-th set, given by Eqs. (|4]) and (jSJ. While evaluating the 
A^pop fitness values, F records on *S the best ^best and 
its corresponding parameter vector set {pi, best, P3, best}- 
The above steps [L.l]- [L.3] are repeated until ^best 
reaches close to 1. In an ideal case, the simulator finds 
{pi,best,P3,best} that yields ^best = 1 with Pc = 1 and 
Pb = 0. The found parameters lead to an algorithm 
equivalent to the original DJ. 

Numerical simulations. - Numerical (Monte-Carlo) 
simulations for n-bit DJ problem are done with increas- 
ing n from 1 to 5. In the simulations, we take A/'pop = 10 
for all n [slj. The free parameters W and Cr are set to 



achieve the best learning efficiency: W is fixed to be 0.1 
for all n and Cr is chosen between 0.05 and 0.1 for each 
n. The results are given in Fig. [3l^a), where we present 
the averaged best fitness ^best^ sampling 1000 trials. It 
is clear to observe that ^best approaches to 1 as iteration 
proceeds. The required stage is just one for all n. This 
implies that our simulator can faithfully learn a single- 
query quantum algorithm for DJ problem, showing ^ ^ 1. 
It is also notable that the found algorithms are equiva- 
lent to, but not exactly the original DJ algorithm: The 
found Ui and are always different, but constitute an 
algorithm solving DJ problem. 

We present a learning probability P(r), defined by the 
probability that the learning is completed before or at 
r-th iteration Here we assume a halting condition 

^best ^ 0.99 to terminate the learning, i.e. the simula- 
tor stops its learning when the condition is satisfied. In 
Fig. [3l^b), we present P(r) for all n, each of which is av- 
eraged by 1000 simulations. We find that P(r) is well 
fitted to an integrated gaussian G(r) = j^^dr' p(r')., 
where probability density p{r) is a gaussian function 



2Ar 



Here, Tc is an average number 
of iterations and Ar is the standard deviation over the 
simulations, which characterize how many iterations are 
sufficient for a statistical accuracy of ^best — 0.99. Note 
that we have finite values of Tc and Ar for all n. The 
probability density p{r) is also drawn in Fig.[3fc), result- 
ing from P(r). 

In Fig. [3jd), we present the graph of Tc versus V^, 
where D is the total number of the control parameters, 
and Tc is the average number of iterations to complete the 
learning. Remarkably, the data are well fitted linearly to 
Tc A\^ + B with A 43 and B -57. Thus, as 
regarding; rQ as a learning time, the learning time is pro- 
portional to the square root of the size of the parameter 
space. Such behavior is completely different from those of 
previous works. Usually, the effectiveness and the conver- 
gence of learning exponentially deteriorates with the size 



of the parameter space (See, for example, Refs. [3l|, l32| 
and their references). To our knowledge, none of the 
previously known methods show such a remarkable con- 
vergence. 

Summary and remarks. - We have presented a method 
for quantum- algorithm design based on machine learning. 
The simulator we have used is a quantum- classical hybrid, 
where the quantum student is being taught by a classical 
teacher. We discussed that such a hybridization is bene- 
ficial in terms of the usefulness and the implementation 
cost. Our simulator is of general purpose and, in prin- 
ciple, every quantum algorithm could be found with it. 
As a case study, we demonstrated that our simulator can 
faithfully learn a single-query quantum algorithm that 
solves Deutsch-Jozsa problem even though it does not 
have to. The found algorithms are equivalent, but not 
exactly equal, to the original Deutsch-Jozsa algorithm 
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FIG. 3: (Color Online) (a) Averaged best fitness ^^est versus 
iteration r. Each data is averaged over 1000 simulations. It is 
observed that ^ly^^t approaches unity as iterating, (b) Learn- 
ing probability P(r) in the halting condition ^best > 0.99, 
sampling 1000 trials. P(r) is well fitted to an integrated 
gaussian (black solid line), G{r) = fl^dr' p{r'). (c) Prob- 
ability density p(r) resulting from P(r) for each n. (d) Graph 
of Tc versus ^/D, where D is the total number of the con- 
trol parameters, and rc is the average number of iterations 
to complete the learning. The data are well fitted linearly to 
rc AVd + B with A ^ 43 and B ^ -57. 



with the fitness 1. 

Even more remarkable was that the learning time is 
proportional to the square root of the size of the param- 
eter space instead of the exponential dependance in the 
classical cases. This result will work in synergy with the 
findings from Ref. jT^ that for quantum algorithms the 
size of the parameter space can be significantly smaller 
than for their classical counterparts. The quantum ma- 
chines are expected to have a double advantage: Not only 
their learning time scales more favorably with the size of 
the space but also this size is smaller to begin with. 

We expect that the proposed method will be used to 
find an essential quantum operation or gate that have 
never been expected crucial in a quantum algorithm, 
and could open a new field of quantum-algorithm design. 
Nevertheless, it is an open question as to whether we can 
observe more improved behaviors in quantum-algorithm 
design when employing a quantum feedback, compared 
with the classical feedback. 
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